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Abstract — We address the problem of securing distributed 
storage systems against eavesdropping and adversarial attacks. 
An important aspect of these systems is node failures over time, 
necessitating, thus, a repair mechanism in order to maintain 
a desired high system reliability. In such dynamic settings, an 
important security problem is to safeguard the system from 
an intruder who may come at different time instances during 
the lifetime of the storage system to observe and possibly alter 
the data stored on some nodes. In this scenario, we give upper 
bounds on the maximum amount of information that can be 
stored safely on the system. For an important operating regime 
of the distributed storage system, which we call the bandwidth- 
limited regime, we show that our upper bounds are tight and 
provide explicit code constructions. Moreover, we provide a way 
to short list the malicious nodes and expurgate the system. 

Index Terms — Byzantine adversary, Distributed Storage, Net- 
work Codes, Secrecy. 

I. Introduction 

Distributed storage systems (DSS) consist of a collection of 
n data storage nodes, typically individually unreliable, that are 
collectively used to reliably store data files over long periods 
of time. Applications of such systems are innumerable and 
include large data centers and peer-to-peer file storage systems 
such as OceanStore fl], Total Recall E) and DHash++ (3) that 
use a large number of nodes spread widely across the Internet. 
To satisfy important requirements such as data reliability and 
load balancing, it is desirable for the system to be designed to 
enable a user, also referred to as a data collector, to download 
a file stored on the DSS by connecting to a smaller number k, 
k < n, nodes. An important design problem for such systems 
arises from the individual unreliability of the system nodes 
due to many reasons, such as disk failures (often due to the 
use of inexpensive "commodity" hardware) or peer "churning" 
in peer-to-peer storage systems. In order to maintain a high 
system reliability, the data is stored redundantly across the 
storage nodes. Moreover, the system is repaired every time a 
node fails by replacing it with a new node that connects to d 
other nodes and download data to replace the lost one. 

Codes for protecting data from erasures have been well 
studied in classical channel coding theory, and can be used 
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Fig. 1. An example of a distributed data storage system under repair. A file 
J 7 of 4 symbols (oi, C12, fci, 62) 6 F 5 is stored on four nodes using a (4, 2) 
MDS code. Node v\ fails and is replaced by a new node V5 that downloads 
(61 + 62), (ai + a2 +61 +62) and (ai + 4a2 + 2b± + 262) from nodes 
1)2, t>3 and V4 respectively to compute and store (ai +a2,a\ +4a2). Nodes 
t>2 , • ■ ■ , v 5 form a new (4, 2) MDS code. The edges in the graph are labeled 
by their capacities. The figure also depicts a data collector connecting to nodes 
1)2 and 1)4 to recover the stored file. 



here to increase the reliability of distributed storage systems. 
Fig. Q] illustrates an example where a (4,2) maximal distance 
separable (MDS) code is used to store a file J 7 of 4 symbols 
(01,02,61,62) G IF5 distributively on n = 4 different nodes, 
vi,...,V4, each having a storage capacity of two symbols. 
The (4, 2) MDS code ensures that a data collector connecting 
to any k = 2 storage nodes, out of n = 4, can reconstruct the 
whole file T. However, what distinguishes the scenario here 
from the erasure channel counterpart is that, in the event of a 
node failure, the system needs to be repaired by replacing 
the failed node with a new one. A straightforward repair 
mechanism would be to add a replacement node that connects 
to k — 2 other nodes, downloads the whole file, reconstructs 
the lost part of the data and stores it. One drawback of this 
solution is the relatively high repair bandwidth, i.e., the total 
amount of data downloaded by the new replacement node. 
For this straightforward repair scheme, the repair bandwidth 
is equal to the size of the file T which can be large in 
general. A more efficient repair scheme that requires less 
repair bandwidth is depicted in Fig. Q] where node V\ fails 
and is replaced by node 1)5. By making node V5 connect to 
d = 3 nodes instead of k = 2, it is possible to decrease 
the total repair bandwidth from 4 to 3 symbols. Note that, 
in the proposed repair solution, v$ does not store the exact 
data that was on v±; the only required property is that the 
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data stored on all the surviving nodes v%, 1*3, U4 and v$ form a 
(4, 2) MDS code. The above important observations were the 
basis of the original work of |4j where the authors showed 
that there exists a fundamental tradeoff between the storage 
capacity at each node and the repair bandwidth. They also 
introduced and constructed regenerating codes as a new class 
of codes that generalizes classical erasure codes and permits 
the operation of a DSS at any operational point on the optimal 
tradeoff curve. 

When a distributed data storage system is formed using 
nodes widely spread across the Internet, e.g., peer-to-peer 
systems, individual nodes may not be secure and may be thus 
susceptible to an intruder that can eavesdrop on the nodes and 
possibly modify their data, e.g., viruses, botnet, etc. In this 
work, we address the issue of securing dynamic distributed 
storage systems, with nodes continually leaving and joining the 
system, against such intruders. The dynamic behavior of the 
system can jeopardize the data by making the intruder more 
powerful. For instance, while eavesdropping on a new node 
during the repair process, the intruder can observe not only 
its stored content but also all its downloaded data. Moreover, 
it allows an adversary to introduce errors on nodes beyond 
his/her control by sending erroneous messages when contacted 
for repair. 

In our analysis, we focus on three different types of in- 
truders: (i) a passive eavesdropper who can eavesdrop on £ 
nodes in the system, (ii) an active omniscient adversary who 
has complete knowledge of the data stored in the system 
and can maliciously modify the data on any b nodes in the 
system, and (iii) an active limited-knowledge adversary who 
can eavesdrop on any I nodes and can maliciously corrupt 
the data on any b nodes among the I observed ones. In the 
last case, the intruder's knowledge about the stored data in 
the system is limited to what can be inferred from the nodes 
he/she is observing. 

We define the secrecy and resiliency capacities of a dis- 
tributed storage system as the maximum amount of informa- 
tion that it can store safely, respectively, in the presence of 
an eavesdropper or a malicious adversary. For these intruder 
scenarios, we derive general upper bounds on the secrecy 
and resiliency capacity of the system. Motivated by system 
considerations, we define an important operation regime that 
we call the bandwidth-limited regime where there is a fixed 
allowed budget for the repair bandwidth with no constraints 
on the node storage capacity. This regime is of increasing 
importance due to the asymmetry in the cost of bandwidth vs. 
storage. For the bandwidth-limited regime, we show that our 
upper bounds are tight and provide explicit constructions of 
capacity-achieving codes. 

The work in this paper is related to the recent work in the lit- 
erature on secure network coding for networks with restricted 
wiretapping sets (5) and networks comprising traitor nodes 
J6). The problem of studying such networks is known to be 
much harder in general than models considering (unrestricted) 
compromised edges instead of nodes. For instance, the work 
of lO implies that finding the secrecy capacity of networks 
with wiretapped nodes is an NP-hard problem. Moreover, non- 
linear coding at intermediate network nodes may be necessary 



for securing networks against malicious nodes as shown in 
J6). The contribution of this paper resides, at a high level, 
in showing that the networks representing distributed storage 
systems have structural symmetry that makes the security 
problem more tractable than in general networks. We lever- 
age this fact to derive the exact expressions of the secrecy 
and resiliency capacities of these systems in the important 
bandwidth-limited regime. Moreover, we present capacity- 
achieving codes that are linear. These codes are characterized 
by a separation property: the file to be stored is first encoded 
for security then stored in the system without any modification 
to the internal operation of the system nodes. An additional 
interesting property of our proposed codes is that, in the active 
adversary case, they permit the identification of a small list 
of suspected nodes guaranteed to contain the malicious ones, 
permitting thus the expurgation of the system. 

The rest of this paper is organized as follows. In Section HU 
we discuss related work on distributed storage systems and 
secure network coding. In Section [HI] we describe the flow 
graph model for distributed storage systems and elaborate on 
the intruder model. We provide a brief summary of our main 
results in Section [IV] In Section [V] we derive an upper bound 
on the secrecy capacity of the system and provide an achiev- 
able scheme for the bandwidth-limited regime. We provide 
a similar analysis for the omniscient and limited-knowledge 
adversary cases respectively in Section |VT] and Section IVHI 
where we find upper bounds on the resiliency capacity and 
construct capacity achieving codes for the bandwidth-limited 
regime. We conclude the paper in Section IVIIII and discuss 
some related open problems. 

II. RELATED WORK 

The pioneering work of Dimakis et al. in J4|, 10, 0, 
demonstrated the fundamental trade-off between repair band- 
width and storage cost in a distributed storage system, where 
nodes fail over time and are repaired to maintain a desired 
system reliability. They also introduced regenerating codes 
as codes that are more efficient than classical erasure codes 
for distributed storage applications. In many scenarios of 
interest, the data is required to exist in the system always 
in a systematic form. This has motivated the study of exact 
regenerating codes |9), iflOl . ifTTI . lfl2l that achieve this goal 
by repairing a failed node with an exact copy of the lost 
data. The construction of exact regenerating codes in |9,| turns 
out to be instrumental in achieving the secrecy and resiliency 
capacity of a DSS in the bandwidth-limited regime. 

In 0, the construction of regenerating codes was linked to 
finding network codes for a suitable network. Network coding 
was introduced in the seminal paper of ifTJl and extends the 
classical routing approach by allowing the intermediate nodes 
in the network to encode their incoming packets as opposed 
to just copying and forwarding it. The literature on network 
coding is now rich in interesting results which can be found 
in references lfl4l and lfT31 . that provide a comprehensive 
overview of this area. 

In this paper, we are interested in securing distributed 
storage systems under repair dynamics, which is a special 
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case of the more general problem of achieving security in 
dynamical systems. A node-based intruder model is natural 
in this setting and is related to the recent work of |[T6l on 
securing distributed storage systems in the presence of a 
trusted verifier and that of Kosut et al. in @ on protecting 
data in networks with traitor nodes. An intruder model that 
can observe and/or change the data on links, as opposed to 
nodes, has been extensively studied in the network coding 
literature. Cai and Yeung introduced in ifTTl . Ifl8l the problem 
of designing secure network codes in the presence of an 
eavesdropper, which was further studied in |fl9l , l20l . 12T1 . 
0. A Byzantine adversary that can maliciously introduce 
errors on the network links was investigated in ll22l . l23l . l24l . 
ll25l . l26l . The problem of error correction in networks was 
also studied by Cai and Yeung in l27l . l28l from a classical 
coding theory perspective. A different approach for correcting 
errors in networks was proposed by Koetter and Kschischang 
in ll29ll . where communication is established by transmitting 
subspaces instead of vectors through the network. The use of 
maximum rank-metric codes for error control under this model 
was investigated in l30l . 

III. Model 
A. Distributed Storage System 

A distributed storage system (DSS) is a dynamic network 
of storage nodes. These nodes include a source node that has 
an incompressible data file T of R symbols, or units, each 
belonging to a finite field F. The source node is connected to n 
storage nodes v\, . . . , v n , each having a storage capacity of a 
symbols, which may be utilized to save coded parts of the file 
T. The storage nodes are individually unreliable and may fail 
over time. To guarantee a certain desired level of reliability, we 
assume that the DSS is required to always have n active, i.e., 
non-failed, storage nodes that are simultaneously in service. 
Therefore, when a storage node fails, it is replaced by a new 
node with the same storage capacity a. The DSS should be 
designed in such a way as to allow any legitimate user or 
data collector, that contacts any k out of the n active storage 
nodes available at any given time, to be able to reconstruct the 
original file T. We term this condition as the reconstruction 
property of distributed storage systems. 

We assume that nodes fail one at a time 1 , and we denote by 
v n+ i the new replacement node added to the system to repair 
the i-th failure. The new replacement node connects then to 
some d nodes, d > k, chosen, possibly randomly, out of the 
remaining active n — 1 nodes and downloads 7 units of data in 
total from them, which are then possibly compressed (if a < 
7) and stored on the node. The data stored on the replacement 
node can be different than the one that was stored on the failed 
node, as long as the reconstruction property of the DSS is 
retained. The process of replenishing redundancy to maintain 
the reliability of a DSS is referred to as the "regeneration" or 

'Multiple nodes failing simultaneously is a rare event. When this occurs, 
the DSS implements an "emergency" recovery process that employs a reserved 
set of trusted nodes, guaranteed not to be compromised. The trusted nodes 
then replace the failed ones by acting as data collectors and downloading data 
from k active nodes. The trusted nodes then consecutively leave the system, 
thus triggering multiple rounds of the repair process. 




Fig. 2. The flow graph model of the DSS 15(4,2, 3) of Fig. [T] when node 
vi fails and is replaced by node U5. Each storage node Vi is represented by 
two nodes x\ n and x l out connected by an edge i^Xn^^out) °f capacity a 
representing the node storage constraint. A data collector DC connecting to 
nodes V2 and 114 is also depicted. 



"repair" process, and we call 7, the total amount of data (in 
symbols) downloaded for repair, the repair bandwidth of the 
system. 

Due to load balancing and "fairness" requirements in the 
system, the repair process is typically symmetric where the 
new replacement node downloads equal amount of data, (3 = 
j/d units, from each of the node participating in the repair 
process. We will adopt the symmetric repair model throughout 
this paper. A distributed storage system V is thus characterized 
as £>(n, k,d), where k < d < n — 1. For instance, the DSS 
depicted in Fig. Q] corresponds to X>(4, 2,3) operating at the 
point (a, 7) = (2,3). 

B. Flow Graph Representation 

We adopt the same model as in J4) where the distributed 
storage system is represented by an information flow graph 
Q. The graph Q is a directed acyclic graph with capacity 
constrained edges. It consists of three kinds of nodes: a single 
source node s, input storage nodes x\ n and output storage 
nodes x l out , and data collectors DC, for i, j <E {1,2,...}. The 
source node s holds an information source S having the file 
J 7 as a special realization. Each storage node 1^ in the DSS 
is represented by two nodes x\ n and x l out in Q. To account 
for the storage capacity of Vi, these two nodes are joined by 
a directed edge (xl n ,x l out ) of capacity a (see Fig. 0. 

The repair process that is initiated every time a failure 
occurs, causes the DSS, and consequently the flow graph, to be 
dynamic and evolving with time. At any given time, each node 
in the graph is either active or inactive depending on whether it 
has failed or not. The graph Q starts with only the source node 
s and the nodes xl n ,...,x" connected respectively to the 
nodes x l out , . . . , x™ ut . Initially, only the source node s is active 
and is connected to the storage input nodes x\ nl . . . ,x™ n by 
outgoing edges of infinite capacity. From this point onwards, 



4 



the source node s becomes and remains inactive, and the n 
input and output storage nodes become active. When a node Vi 
fails in a DSS, the corresponding nodes x\ n and x % out become 
inactive in Q. If a replacement node Vj joins the DSS in the 
process of repairing a failure and connects to d active nodes 
Vi 1 , . . . , Vi d , the corresponding nodes x\ n and x J out with the 
edge (x J in ,x J out ) are added to the flow graph Q, and node 
x\ n is connected to the nodes x^ ut , . . . , x l £ ut by incoming 
edges of capacity (3 = j/d units each. A data collector is 
represented by a node connected to k active storage output 
nodes through infinite capacity links enabling it to download 
all their stored data and reconstruct the file T. The graph 
Q constitutes a multicast network with the data collectors as 
destinations. An underlying assumption here is that the flow 
graph corresponding to a distributed storage system depends 
on the sequence of failed nodes. As an example, we depict in 
Fig. [2] the flow graph corresponding to the DSS T>{A, 2, 3) of 
the previous section (see Fig. [TJ when node V\ fails. 

Let V be the set of nodes in the flow graph Q. A cut C(V, V) 
in the flow graph separating the source s from a data collector 
DC; is a partition of the node set of Q into two subsets V C V 
and V = V \ V, such that s E V and DQ <E V. We say that 
an edge (ni,ri2) belongs to a cut C(V,V) if n\ £ V and 
ri2 G V. The value of a cut is the sum of the capacities of the 
edges belonging to it. 



C. Intruder Model 

We assume the presence of an illegitimate intruder in the 
DSS who can eavesdrop on some of the storage nodes, and 
possibly alter the stored data on some of them in order to 
sabotage the system. We characterize the power of an intruder 
by two parameters £ and b, where I denotes the number of 
nodes that the intruder can eavesdrop on, and b denotes the 
number of nodes it can control by maliciously corrupting 
its data. We distinguish among three categories of intruders: 
a passive eavesdropper "Eve", an active omniscient adver- 
sary "Calvin", and an active limited-knowledge adversary 
"Charlie". We always assume that all the data collectors and 
intruders have the complete knowledge of the storage and the 
repair scheme implemented in the system. 

a) Passive Eavesdropper: We assume that the eavesdrop- 
per Eve can access up to I, £ < k, nodes of her choice among 
all the storage nodes, i>i,V2,---, possibly at different time 
instances as the system evolves. Eve is passive and can only 
read the data on the observed I nodes without modifying it, i.e., 
b = 0. In the flow graph model, Eve is an eavesdropper that 
can access a fixed number £ of nodes chosen from the storage 

input nodes xj n , xf n , Notice that while a data collector 

observes the output storage nodes, i.e., the data stored on the 
nodes it connects to, Eve, has access to the input storage nodes, 
and thus can observe, in addition to the stored data, all the 
messages incoming to these nodes. As a result, Eve can choose 
some of the compromised £ nodes to be among the initial n 
storage nodes, and/or, if she deems it more profitable, she can 
wait for certain failures to occur and then eavesdrop on the 
replacement nodes by observing its downloaded data. 



b) Active Omniscient Adversary: The active adversary 
Calvin is omniscient l24l . i.e., he knows the file T and the 
data stored on all the nodes. Moreover, Calvin can control 
b nodes in total, where 2b < k, that can include some of 
the original nodes vi, . . . , v n , and/or some replacement nodes 
v n+ i, .... Calvin can maliciously alter the data stored on the 
nodes under his control. It can also send erroneous outgoing 
messages when contacted for repair or reconstruction. In the 
flow graph, this corresponds to controlling a set of b input 
nodes {x^, x l - 2 n , . . . , x l f n } and the corresponding output nodes 

Srj.il _*a ib \ 

X^outJ -^outi • • • 1 ^outl- 

c) Active Limited-knowledge Adversary: The active ad- 
versary Charlie is not omniscient but has limited knowledge 
about the data stored in the system. In particular, he has a 
limited eavesdropping capability £ not sufficient enough to 
know all the stored data. In addition, Charlie can control b 
nodes of his choice and maliciously corrupt their data. In 
distributed storage systems, an intruder controlling a node will 
also observe its data. Therefore, we assume that b < £, and that 
these b nodes are a subset of the £ eavesdropped nodes. In the 
flow graph, this corresponds to eavesdropping on some I input 
nodes {x^, . . . , x\ l n } and controlling a subset of size b of these 
nodes and the corresponding output nodes. A similar model 
was studied in |23l . lF24l . ll25l where the authors consider a 
limited-knowledge adversary that can eavesdrop and control 
edges rather than nodes in multicast networks. 

IV. Results 

The primary goal of this work is to secure distributed stor- 
age systems with repair dynamics in the presence of different 
types of intruders: passive eavesdropper, active omniscient 
adversary and active limited-knowledge adversary. We address 
the following issues: 

• In the case of a passive eavesdropper, we study the 
secrecy capacity C s of the DSS, i.e., the maximum 
amount of data that can be stored on the DSS and 
delivered to a legitimate data collector without revealing 
any information about the data to the intruder. 

• In the case of an active adversary, we study the resiliency 
capacity C r of the DSS, i.e., the maximum amount of 
data that can be stored on the DSS and reliably made 
available to a legitimate data collector. 

For a DSS with symmetric repair, we provide upper bounds 
on the secrecy capacity and resiliency capacity. These bounds 
are maximized for the choice of repair degree d = n — 1. 
In this case, we provide explicit coding schemes that can 
achieve these bounds in the bandwidth-limited regime. Our 
results are summarized in Table Q] We also show that for the 
active adversary controlling b nodes, our capacity achieving 
schemes can identify a list, of size at most 26 nodes, that is 
guaranteed to contain the malicious nodes. Thus, the system 
can be expurgated of these corrupt nodes, and thereby its 
resiliency to active adversaries is rejuvenated. 

The upper bounds in Table U are based on cut arguments 
over the information flow graph representing the DSS |4|- Note 
that when there is no intruder, i.e., £ = b = 0, all the upper 
bounds in the second column of the Table|T]collapse to the DSS 
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Adversary Model 


Upper bound 

l = df3 


Bandwidth limited regime (T) 
d = n- 1,48 =T 


Passive eavesdropper (I < k, b = 0) 


C s {a, 7) < E«*U+i min{(d - i + l)/3, a} 


Cf L (T) = Ef !=4+1 (n-i) J 9 


Active omniscient adversary (I = k, 2ft < k) 


Cr («, 7) < Ef= 2b +i min{(c! - i + I)j8, a} 


^ Si (r) = E"=2 i ,+i(^-*)/3 


Active limited-knowledge adversary^, b < £) 


Cr (a, 7) < E«Wl min{(d - i + a} 


c r SL (r) = E- =i , + i(™-0/3 



TABLE I 

Summary of our capacity results for a DSS X>(n, k, d), with a units of storage capacity at each node and 7 = d/3 repair bandwidth. 

AN ADVERSARY IS CHARACTERIZED BY TWO PARAMETERS: I, THE NUMBER OF NODES IT CAN EAVESDROP ON, AND ft, THE NUMBER OF NODES IT CAN 
CONTROL. C s AND C r DENOTE THE SECRECY CAPACITY AND RESILIENCY CAPACITY, RESPECTIVELY. T IS THE UPPER LIMIT ON THE REPAIR 
BANDWIDTH FOR THE BANDWIDTH-LIMITED REGIME. NOTE THAT IF THE CONDITIONS ON i, ft SPECIFIED IN THE FIRST COLUMN ARE NOT SATISFIED, 

THEN Cs,Cr ARE EQUAL TO ZERO 



capacity M = Y^i=i min{(d — i + l)/3, a} which was derived 
in the original work of |4|. The upper bound on the secrecy 
capacity C s , for the case of a passive eavesdropper can be 
explained intuitively by recognizing that when the DSS knows 
the identity of the I compromised nodes it can discard them 
and avoid using them for storage. Hence, in the expression 
of the upper bound on C s , we see a loss of I terms in the 
summation as compared to the capacity with no intruder. 

The upper bound on the resiliency capacity C r , for the 
case of an active omniscient adversary, is similar to the one 
derived in (5) and can be regarded as a network version of 
the Singleton bound: a redundancy of 2b nodes is needed in 
order to correct the adversarial errors on b nodes. Whereas, 
a feasible strategy for the limited-knowledge adversary is to 
delete the data stored on the b nodes it controls rendering them 
useless resulting in the corresponding upper bound. Rigorous 
proofs of these results will be provided in the coming sections. 

To get more insight into the above results for the bandwidth- 
limited case, we consider an asymptotic regime for the DSS 
where the number of nodes goes to infinity whereas the 
parameters k, £ and b are kept constant. We compute the ratios 
Cf L /M and C^ L /M, where M is the capacity of the DSS in 
the absence of any intruder. This ratio for the secrecy capacity 
is, 



M 



(1) 



as n — ¥ 00. Similarly, for the resiliency capacities, we have 
for omniscient adversary, 

C? L (T) _ 2b 

k 



M 



(2) 



And for limited-knowledge adversary, 



C? L (T) 
M 



>4 



(3) 



Note that these asymptotic ratios are reminiscent of the ca- 
pacity of the classical wiretap channel iTJlll in the case of a 
passive eavesdropper (fTJ, the Singleton bound ||32l in the case 
of omniscient adversary (f2j), and the capacity of the erasure 
channel ll33l for the case of limited-knowledge adversary (0). 



V. Passive Eavesdropper 

In this section, we consider a distributed storage system 
T>(n,k,d) in the presence of a passive intruder "Eve". As 
described in Section Hill Eve can eavesdrop on any I < k 
storage nodes 2 of her choice in order to learn information 
about the stored file. However, Eve cannot modify the data 
on these nodes. We assume that Eve has complete knowledge 
of the storage and repair schemes implemented in the DSS. 
Next, we define the secrecy capacity of a DSS as the maximum 
amount of data that can be stored on a DSS under a perfect 
secrecy requirement, i.e., without revealing any information 
about it to the eavesdropper. 

A. Secrecy Capacity 

Let S be a random variable uniformly distributed over 
representing the incompressible data file of size R symbols at 
the source node, which is to be stored on the DSS. Thus, we 
have H(S) = R (in base log g ). Let V in := {x\ n , xf n , . . . } 
and V ou t := {x^ ut , x^ ut , . . . } be the sets of input and output 
storage nodes in the flow graph, respectively. For each storage 
node Vi, let Di and C% be the random variables representing its 
downloaded messages and stored content respectively. Thus, 
Ci represents the data observed by a data collector DC when 
connecting to node Vi. If is compromised while joining 
the DSS, Eve will observe all its downloaded data Di, with 
H(Di) < 7, and not only what it stores. 

Let V D a ut be the collection of all subsets of V ou t of cardinal- 
ity k consisting of the nodes that are simultaneously active, 
i.e., not failed, at a certain instant in time. For any subset B of 



V out , define C B := {C, 



£ B}. Similarly for any subset 



E of Vi n , define De '■= {Di : x\ n <E E}. The reconstruction 
property at the data collector can be written as 



H(S\C B ) = VBeV a w 



(4) 



and the perfect secrecy condition implies 

H(S\D E ) = H(S) \/E C V in and \E\ < I. (5) 

2 When Eve observes I > k the secrecy capacity of the system is trivially 
equal to zero since Eve can implement the data collector's scheme to recover 
all the stored data. 
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Given a DSS T>(n, k, d) with £ compromised nodes, its 
secrecy capacity, denoted by C s (a, 7), is then defined to be the 
maximum amount of data that can be stored in this system such 
that the reconstruction property in (0]i and the perfect secrecy 
condition in (O are simultaneously satisfied for all possible 
data collectors and eavesdroppers, i.e., 

C s (a, 7):= sup H(S), (6) 

H(S\C B ) = VB 
H(S\D E ) = H(S) VE 

where B g V a a ut , E C V in and |£;| < £. 

B. Special Cases 

Before we proceed to the general problem of determining 
the secrecy capacity of a DSS, we analyze two special cases 
that shed light on the general problem. 

1) Static Systems: A static version of the problem studied 
here corresponds to a DSS with ideal storage nodes that do 
not fail, and hence there is no need for repair in the system. 
The flow graph of this system constitutes then a well-known 
multicast network studied in network coding theory called 
the combination network ifTTl Chap. 4]. Therefore, the static 
storage problem can be regarded as a special case of wiretap 
networks 1T81 . 120) . or equivalently, as the erasure-erasure 
wiretap-II channel studied in (34). The secrecy capacity for 
such systems is equal to (k — t)a, and can be achieved using 
either the nested MDS codes of 11341 or the coset codes of 
GO), EQ. 

Even though the above proposed solution is optimal for the 
static case, it can have a very poor security performance when 
applied directly to dynamic storage systems experiencing 
failures and repairs. For instance, consider the straightforward 
way of repairing a failed node by downloading the whole file 
and regenerating the lost data. In this case, if Eve observes the 
new replacement node while it is downloading the whole file, 
she will be able to reconstruct the entire original data. Hence, 
no secrecy scheme will be able to hide any part of the data 
from Eve, and the secrecy rate would be zero. 

The case of static systems highlights the new dimension that 
the repair process brings into the secrecy picture of distributed 
storage systems. The dynamic nature of the DSS renders it 
intrinsically different from the static counterpart making the 
repair process a key factor that should be carefully designed 
in order not to jeopardize the whole stored data. 

2) Systems Using Random Network Coding: Using the flow 
graph model, the authors of |4) showed that random linear 
network codes over a large finite field can achieve any point 
(a, 7) on the optimal storage-repair bandwidth tradeoff curve 
with a high probability. Consider an example of a random 
linear network code used in a compromised DSS X>(4, 3, 3) 
which stores a file of size R = 6 symbols with (3 = 1, i.e., 
7 = dp = 3, and a = 3. From J4), it can be shown using the 
max-flow min-cut theorem that the maximum file size that can 
be stored on this DSS is equal to 6 symbols. In this case, each 
of the initial nodes V\ , . . . , V4 store 3 independently generated 
random linear combinations of the 6 information symbols. 
Assume now that node V4 fails (see Fig. [3]l and is replaced 




Fig. 3. The DSS 25(4,3,3) with (0,7) = (3,3), i.e., = 1. Eve can 
observe 1 = 2 nodes. Node V4 fails and is replaced by node 1)5, which fails 
in turn after some time and is replaced by node vg. Nodes 115 and vg are 
compromised and shown with broken boundaries. If random network coding 
is used and Eve observes nodes D5 and vq during repair, it will be able to 
decode all the stored data with a high probability. 



by a new node v$ that connects to v%,V2,V3 and downloads 
from each /3 = 1 random linear combination of their stored 
data. Now suppose that node V5 fails after some time and is 
replaced by node vq in a similar fashion. If I = 2 and Eve had 
accessed nodes 1)5 and vq while they were being repaired, it 
would observe 6 random linear equations of the data symbols. 
Since the underlying field is typically of large size, the 6 linear 
equations observed by Eve are linearly independent with high 
probability. Hence, she will be able to reconstruct the whole 
file, and the secrecy rate here is equal to 0. Later in Example [3] 
we present a scheme that achieves a secrecy rate of 1 unit for 
this DSS. 

While random network codes are appealing for use in 
distributed storage systems due to their decentralized nature 
and low complexity, the above analysis shows that this may 
not always be the case for achieving security. This is also in 
contrast with the case of multicast networks where an intruder 
can observe a fixed number of edges instead of nodes |[T8) . 
wherein, random network coding performs as good as any 
deterministic secure code |2"T) . 

C. Results on Passive Eavesdropper 

We present here our two main results for the compromised 
DSS with passive eavesdropper: 

Theorem 1: [Secrecy Capacity Upper Bound] For a dis- 
tributed storage system T>(n, k, d), with £ < k compromised 
nodes, the secrecy capacity is upper bounded by 

k 

C s (a, 7 ) < Y, min{(d-i + l)P,a}, (7) 
i=e+i 

where (3 = 7/d 

In the bandwidth-limited regime, we have a constraint on 
the repair bandwidth 7 < T, while no constraint is imposed 
on the node storage capacity a. The secrecy capacity in this 
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Fig. 4. The flow graph of the DSS 75(4, 3, 3) with (a, 7) = (3, 3), = 1 
and £ = 2. Node 113 fails and is replaced by node 115. Nodes v\,V2 
are compromised to Eve and are shown with broken boundaries. A data 
collector DC connects to nodes vi, V2 , ^5 to retrieve the data file. The 
data collector can get at most one unit of information securely on the path 



„4 4 „5 
u out> " 



x^ ut ,DC) which is not observed by Eve. 



regime is thus defined as 



cf L (r) 



< 



sup C s (a,7) 

7 < r 

a > 

sup l)/3- 

7 < r j=f+i 



(8) 



(9) 



The last inequality follows from Theorem Q] by setting a = V. 
When the parameter d is a system design choice, the maximum 
in the above optimization is attained at d* = n — 1. In 
Section [V-DI we demonstrate a scheme that achieves this upper 
bound, thereby establishing the following theorem. 

Theorem 2: [Secrecy Capacity: Bandwidth-Limited 
Regime] For a distributed data storage system T>(n, k, d) 
with d = n — 1 and I < k compromised nodes, the secrecy 
capacity in the bandwidth-limited regime is given by 



cf L (r) 



k 

E 

i=e+i 



(n-i)8, 



where 8 = — Ur and can be achieved for a node storage 
capacity a = T, 

Before we proceed to prove the above theorems, we consider 
an example that gives insights into the proof techniques. 

Example 3: Consider again the DSS T>(A, 3, 3) operating at 
a = 3, 8 = 1 and £ = 2 of Section IV-B21 We show first that 
the upper bound on the secrecy capacity of this system is 1 as 
given by Theorem [T] and then provide a scheme that achieves 
it. 

To obtain the upper bound on the secrecy capacity, consider 
the flow graph of this DSS shown in Fig. [4] where nodes v\ 
and V2 are compromised and observed by Eve. Suppose that 
node V3 fails and is replaced by v$ that downloads 6 = 1 
unit of information from each of the d = 3 nodes vi 1 v 2l v^. 
We focus now on a data collector that connects to the three 
nodes V\ , v 2 and u 5 to reconstruct the source file. Even if the 
source node s and the data collector knew the location of the 



eavesdropper, it can get at most one unit of secure information 
by ignoring all the information received from the compromised 
nodes. The data can only be conveyed securely through the 



path (s, xf n , x\ uU x\ n , x 5 out ,DC), that has a "bottleneck" edge 



(x 



4 

out ) ' 



J with capacity 



1 unit. Since our analysis is 



based on a worst case scenario, this gives an upper bound of 
1 unit on the secrecy capacity. This bound can be reinterpreted 
as taking the minimum value of a cut separating the source 
s from any data collector in the flow graph after deletion of 
any two nodes. This argument can be generalized to any DSS 
T)(n,k,d) by finding an upper bound on the value of the 
min-cut in the flow graph after deleting £ nodes. Thus, we 
obtain the upper bound of Theorem Q] whose detailed proof is 
provided in Appendix lAl 

Before we provide a coding scheme that achieves the previ- 
ous upper bound, we define the nested MDS codes l34l which 
will be an important building block in our code construction. 

Definition 4 (Nested MDS Codes): An (n, k) MDS code 
with generator matrix G is called nested if there exists a 

G 

positive integer fco < k such that G = r , 

Gi 

dimensions (fco x n), itself is a generator matrix of an (n, fco) 
MDS code. 

Our proposed capacity-achieving code is depicted in Fig. [5] 
and consists of the concatenation of an outer nested MDS code 
with a special inner repetition code that was introduced in (9) 
for constructing exact regeneration codes. Let S G ¥ q denote 
the information symbol that is to be securely stored on the 
system and K, = [K\ . . . K5] be a vector of independent 
random keys each uniformly distributed over ¥ q . The MDS 



with G\, of 



coset code is chosen to be a nested MDS code 

Gr 



341 with its 



generator matrix given by G 



G, 



where 



G 



K 



G s 



1 1 

10 10 

10 10 

1 1 

1 1 

[100000] 



and 



Note that the matrix G 



Gk 
G s 



a generator of a (6, 6) 



MDS code and the sub-matrix Gk is a generator of an (6, 5) 
MDS code (fco = 5). Hence, the code generated by G is a 



nested MDS code. Set, Z 
X given by 



S + J2i=i K<> men tne codeword 



X = [ JC s ] 



Gk 
G s 



(10) 



can be written as X = [ Z K\ ... K<~, ] . The encoded 
symbols Z, K\ , . . . , are then stored on the nodes i)\ , . . . , V4 
as shown in Fig. |5j following the special repetition code of 
Rashmi et al 15), which we henceforth refer to as RSKR- 
repetition code. 

In the RSKR-repetition code used here, nodes V\,...,v± 
store respectively {Z, K u K 2 }, {Z,K 3 ,K 4 }, {Kx,K 3 ,K 5 } 
and {K 2 , K4, K^}. Since d = 3, in the case of a failure 
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the new replacement node contacts all the 3 remaining active 
nodes in the system and recovers an exact copy of the lost 
data. For example, when node vi fails the new replacement 
node connects to nodes V2 , W3 and V4 and downloads the 
symbols Z, K\ and K2 from each, respectively. It can also 
be checked that a data collector connecting to any 3 nodes 
observes all the symbols Z, K\ , . . . , K$ and hence can decode 
the information symbol S as S = Z — Y?,i=i However, 
an eavesdropper accessing any two nodes will observe some 
subset of 5 symbols out of 6, and therefore cannot obtain any 
information about S. 



Random keys 



Information 
symbol 

S 



MDS 

coset code 



Z.K, K s 



z = s + Y.U k, 



Node vi 




Z Ki K 2 


Node U2 


Z A"( Ki 


Node u 3 


Ki A' 3 K e 








Node V4 




Kl Ki A'r, 


RSKR-repetition code 



Fig. 5. A schematic representation of the optimal code for the DSS 
X>(4, 3, 3), operating at (a, 7) = (3, 3) with 1 = 2, that achieves the secrecy 
capacity of 1 unit. The information symbol S and 5 independent random keys 
are mixed appropriately using an MDS coset code. The encoded symbols are 
then stored on the DSS using the RSKR-repetition code. An eavesdropper 
observing any £ = 2 nodes cannot get any information about the stored 
symbol S. 

In the following section, we provide a generalization of the 
code in this example, and show that it achieves the secrecy 
capacity of DSS for d = n—1 in the bandwidth-limited regime, 
thus proving Theorem 

D. Secrecy Capacity in the Bandwidth-Limited Regime 

The special cases studied in Section IV-BI pointed out that 
the main difficulty in determining the secrecy capacity of 
distributed storage systems is due to its dynamic nature. We 
will demonstrate that in the bandwidth-limited regime for 
d = n — 1, with a careful choice of code, it is possible to 
transform the problem of secrecy over a dynamic DSS into 
a static problem of secrecy over a point to point channel 
equivalent to the erasure-erasure wiretap channel-II in [34]. 
Then, we show that using nested MDS codes at the source 
one can achieve the secrecy capacity of the equivalent wiretap 
channel. 

Our approach builds on the results of (9l where the authors 
constructed a family of exact regenerating codes for the DSS 
T>(n,k,d) with d = n — l,a = df3. The "exact" property 
of these codes allows any repair node to reconstruct and 
store an identical copy of the data lost upon a failure. The 
code construction in J9) consists of the concatenation of an 
MDS code with the RSKR-repetition code. This construction 
is instrumental for obtaining codes that can achieve the secrecy 
capacity by carefully choosing the outer code to be a nested 
MDS coset code as was done in Example [5] 

For simplicity, we will explain the code for /3 = 1, i.e., T = 
n — 1. For any larger values of V, and in turn of j3, the file can 
be split into chunks, each of which can be separately encoded 
using the construction corresponding to f3 = 1. Since the DSS 



Node vi X\ X2 X3 



Xd 



Node v 2 X\ Xd+\ Xd+2 



X2d-1 



Nodev 3 X2 x d+ i x 2 d 



%3d-3 



I : 



Node v„ 



Xd X 2 d-1 X 3d -3 



X() 



Fig. 6. The structure of the RSKR-repetition code of Rashmi et al (9) for 



n storage nodes, 



l,/3 



1 and 



n(n— 1) 



The RSKR- 



repetition code stores 2 copies of each coded symbol, i.e., the total number 
of stored symbols is nd = 29. 



is operating in the bandwidth-limited regime with no constraint 
on the node storage capacity, we choose a = T. From HJ, we 
know that for a DSS T>(n, k,d = n — 1) with a = n— 1,/3 = 1 
the capacity in the absence of an intruder (£ = 0) is M = 
~ Let R := \\^ k i= £ +1 {n — i) be the maximum 
number of information that we could store securely on the 
DSS, and 6 := Let S = (s u . . . , s R ) € denote the 

information file and K = (Ki, . . . ,Km-r) S F^ - ^- denote 
M — R independent random keys each uniformly distributed 
over F„. Then, the proposed code consists of an outer (9, M) 



nested MDS code (see ([Tol l) which takes S and K, as an input 
and outputs X = (xi, . . . , xg), as, 



X = [ K S ] 



Gk 
G s 



where, G 



Gk 
G s 



is a generator matrix of a (0, M) MDS 



code such that Gk itself is a generator matrix of a (0, M — 
R) MDS code. This outer (6>, M) nested MDS code is then 
followed by an inner RSKR-repetition code which stores the 
codeword X on the DSS following the pattern depicted in 
Fig. [6] 

The RSKR-repetition codes were introduced in (9j as a 
method for constructing exact regenerating codes for a dis- 
tributed storage system. These codes consist of "filling" the 
storage nodes v\ , . . . , v n successively, by repeating "verti- 
cally" {i.e, across all the nodes) the data stored "horizontally" 
(i.e., on a single storage node), as shown in Fig. [6] This 
procedure can be described using an auxiliary complete graph 
over n vertices U\, . . . ,u n that consists of 9 edges. Suppose 
the edges are indexed by the coded symbols xi,...,x$- The 
code then consists of storing on node Vi the indices of the 
edges adjacent to vertex m in the complete graph. As a result, 
the RSKR-repetition code has a special property that every 
coded symbol x% is stored on exactly two storage nodes, and 
any pair of two storage nodes have exactly one coded symbol 
in common. This property along with the fact that the repair 
degree d = n — 1, enables the exact repair of any failed node 
in the DSS as it was explained in Example [3] 

The use of the RSKR-repetition code transforms the dy- 
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namic storage system into a static point-to-point channel as 
explained below. Notice first that since T — a = n — 1, all the 
data downloaded during the repair process is stored on the new 
replacement node without any further compression 3 . Thus, 
accessing a node during repair, i.e., observing its downloaded 
data, is equivalent to accessing it after repair, i.e., only observ- 
ing its stored data. Second, the RSKR-repetition code restore 
the replacement node with an exact copy of the lost data. 
Therefore, even though there are failures and repairs, the data 
storage system looks exactly the same at any point of time: 
any data collector downloads M symbols out of x\, . . . , xg 
by contacting k nodes, and any eavesdropper can observe 
H = Yli=i(d — * + 1) = M — R symbols. Thus, the system 
becomes similar to the erasure-erasure wiretap channel-II of 
parameters (9,M,p) 4 . Therefore, since the outer code is a 
nested MDS code, from l34l we know that it can achieve the 
secrecy capacity of the erasure-erasure wiretap channel which 
is equal to M — /i. Hence for the DSS, our codes achieve the 
secrecy rate of 

k 

M - (M — R) = R= ( n - 

i=t+l 

This rate corresponds to f) = 1. For the general case when 
13 = T/(n — 1), the total secrecy rate achieved is, 

k 

i=t+l 

thus completing the proof of Theorem |2 

VI. Active Omniscient Adversary 

In this section we study distributed storage systems in the 
presence of an active adversary "Calvin" that can control up to 
b nodes. Calvin can choose to control any b nodes among all 
the storage nodes, vi, V2 , ■ ■ ■ , and possibly at different time 
instances as the system evolves in time due to failures and 
repairs. Moreover, Calvin is assumed to be omniscient (I = 
k), so he knows the source file J 7 . Moreover, since he has 
complete knowledge of the storage and repair schemes, he 
knows the content stored on each node in the system. Under 
this setting, we define the resiliency capacity of a DSS as the 
maximum amount of data that can be stored on the DSS and 
delivered reliably to any data collector that contacts any k 
nodes in the system. 

Example 5: Consider again our example of the DSS 
2?(4, 3, 3) with a = 7 = 3. Assume that there is an omniscient 
active adversary Calvin that can control one storage node, i.e., 
b = 1, and can modify its stored data and/or its messages 
outgoing to data collectors and repair nodes. 

A first approach for finding a scheme to reliably store data 
on this DSS would be to use the results in the network coding 
literature EH, |27), |28), |29) on the capacity of multicast 

3 This corresponds to the Minimum Bandwidth Regenerating (MBR) codes 
described in (4). 

4 In the erasure-erasure wiretap channel-II of parameters (8,M,fi), the 
transmitter sends 8 symbols through an erasure channel to a legitimate receiver 
that receives M symbols. The eavesdropper can observe any /i symbols out 
of the transmitted M [34). 
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Fig. 7. A coding scheme for storing 1 bit reliably on the DSS ©(4,3.3) with 
a = 3 bits and = 1, in the presence of an omniscient adversary Calvin 
who controls b = 1 node. 



networks in the presence of an adversary that can control 
t edges of unit capacity each. It is shown there that the 
resiliency capacity of these networks is equal to O — 2t, where 
f2 is the capacity of the multicast network in the absence 
of the adversary. This resiliency capacity can be achieved by 
overlaying an error-correction code such as a Maximum Rank 
Distance (MRD) code ||2T1 on top of the network at the source. 
This approach turns out to be not very useful here. In fact, the 
capacity in the absence of Calvin is 6 (see H), and 6=1 
corresponds to t = a = 3. Hence, the above approach will 
achieve a storage rate of 6 — 2t = 0. 

We now give a coding scheme that can reliably store 1 
bit of information for the DSS. Later, we show that this is 
also the best that can be done, i.e., the resiliency capacity of 
this DSS is equal to 1 unit. The proposed code is formed by 
concatenating a (6, 1) repetition code with an RSKR-repetition 
code as shown in Fig [7] The repair process is that of the 
RSKR-repetition codes described in Section IV-DI When a 
node fails, the replacement node recovers the lost bits by 
downloading the bits with same indices from the remaining 
three active nodes. 

Any data collector contacting three nodes will observe 9 
bits. In the static case, when no failure or repair occur, only 
3 bits (the ones stored on the compromised node) among the 
9 bits observed by the data collector may be erroneous. In 
that case, the DC can perform a majority decoding to recover 
the information bit. However, in the dynamic model, the DC 
can receive up to 5 erroneous bits. To show how this may 
occur, assume that the DSS is storing the all-zero codeword, 
i.e., Xi = for i = 1,...,6, in Fig. [7] corresponding to 
the message m = 0. Suppose that node V\ is the one that 
is compromised and controlled by the adversary Calvin as 
shown in Fig. [8] Assume that Calvin changes all the 3 stored 
bits X2, X3) on node v\, from (0,0,0) to (1,1,1) and 
also sends the erroneous bit "1" whenever v\ is contacted 
for repair. Now suppose that node v 2 fails and it is replaced 
by node U5 which, based on the RSKR-repetition structure, 
downloads bits x\ = 1,X4 = and x$ — from nodes 
Vi , V3 and V4 respectively. Suppose also that, after some period 
of time, node W3 fails and is replaced by node vq which 
downloads bits X2 = 1,^4 = and x§ = from nodes 
vi , V4 and V5 respectively. An important point to note here 
is that our repair scheme is fixed and is based on the RSKR- 
repetition structure irrespective of the possible errors in the 
bits downloaded during the repair process. As a result a data 
collector that contacts nodes V\ , v$ and v§ observes the data 
as shown in the table in Fig. [8] which includes 5 errors. 
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Node Vi 1 11 





Node v 5 


1 




Node i;6 


1 



DC observation 




Fig. 8. Node vi with broken boundary is compromised and controlled by an 
omniscient adversary Calvin. Nodes V2 and v$ fail, and are replaced by nodes 
vs , vq respectively. The all-zero codeword corresponding to message m = 
is stored on the DSS. The Data collector DC connecting to nodes v\, t's and 
Vfj observes a total number of 9 bits out of which 5 bits are erroneous and 
equal to "1" as shown in the table above. 



In a worst case scenario, Calvin will be able to corrupt all 
the bits in the DSS having the same indices as the bits stored 
on the nodes it controls (here the bits with labels xi,x 2 and 
X3). Therefore, Calvin can introduce at most 5 erroneous bits 
on a collection of k = 3 nodes which may be observed by a 
data collector. In this case, a majority decoder, or equivalently 
a minimum Hamming distance decoder, will not be able to 
decode to the correct message. 

To overcome this problem, we exploit the fact that Calvin 
controls only one node, so he can introduce errors only in 
specific patterns, to design a special decoder that will always 
decode to the correct message m irrespective of Calvin's 
adversarial strategy. In fact, for any possible choice of the 
compromised node, one of the following four sets T\ = 
{x4,x 5 ,x 6 },T 2 = {x 2 ,x 3 ,X6},T 3 = {xi,x 3 ,x 5 } and T 4 = 
{xi, 22, £4} is a trusted set that only contains symbols that 
were not altered by Calvin. For example, when Calvin controls 
i>i, the trusted set is T\. The proposed decoder operates in 
the following way. First, it finds a set T* 6 {Ti,...,T 4 } 
whose elements all agree to either or 1. Then, it declares 
accordingly that message m = or m = 1 was stored. This 
decoder will always decode to the correct message since each 
set Ti intersects with every other set Tj in exactly one 

symbol and one of them is a trusted set. Therefore each set 
Ti contains at least one symbol which is unaltered by Calvin. 
Thus, if all the symbols in Ti agree, they will agree to the 
correct message. 



A. Results on Omniscient Adversary 

In J6), the resiliency capacity of unicast networks with a 
single compromised node was analyzed and a cut-set upper 
bound was derived. In the following, Theorem [6] generalizes 
the bound in J6] for the case of distributed storage systems, 
where b > 1 nodes are controlled by an omniscient adversary. 

Theorem 6: [Resiliency Capacity Upper Bound] Consider 
a distributed storage system DSS T>(n, k,d). If an omniscient 
adversary controls any b > 1 nodes, with 2b < k, the resiliency 
capacity C r (a,7) is upper bounded as, 

k 

C r (a,-y) < min{(d-i+ 1)0,0}, (H) 

i=2b+l 

where /3 = f/d. If 2b > k, then C r (a, 7) = 0. 

This bound is a network version of the Singleton bound and 
is obtained by computing the value of certain cuts in the flow 
graph of the DSS after the deletion of 2b nodes. The detailed 
proof of the above theorem is given in Appendix [B] 

The resiliency capacity in the bandwidth-limited regime is 
defined as 

C? L (T):= sup C r (a,7), 

7 < r 

a > 

where V is the upper limit on the total repair bandwidth. We 
again note that if the parameter d is a system design choice, 
the upper bound of Eq. (fTTT i in the bandwidth-limited regime 
is maximized for d = n — 1. In the following section we 
exhibit a scheme that achieves this upper bound. This result 
is summarized in Theorem [7] 

Theorem 7: Consider a distributed storage system 
V(n,k,d = n — 1) operating in the bandwidth-limited 
regime. If an omniscient adversary controls b nodes, with 
2b < k, the resiliency capacity of the DSS is given by 

k 

C? L {V) = (""OA (!2) 

i=26+l 

where B = -^—r and can be achieved for a node storage 

" n— 1 & 

capacity a = T. If 2b > k, then C ? Bi (r) = 0. 

B. Resiliency Capacity in the Bandwidth-Limited Regime 

Similar to the proof of Theorem [2] it suffices to show the 
achievability for B — 1, i.e., T = n — 1. In this case, our 
capacity achieving code uses a node storage capacity a = n—1 
symbols. 

The code has a similar structure to the scheme used in 
Section [V] for the case of a passive adversary and is a gener- 
alization of the code used in Example The (6, 1) repetition 
code in the example is replaced by an (9, R) MDS code 

where R := C r (n — 1) = J2'i=2b+i( n _ *) an d ^ = "^""^ ■ 
In the second layer, the output of the MDS code is stored 
on the DSS following the RSKR-repetition structure as in 
Fig As explained in Example [5] node failures are repaired 
using the RSKR-repetition structure (also see Section [V] for 
additional details) irrespective of the possible errors introduced 
by Calvin. Notice that the MDS code used here has a rate 
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lower then the one used in the passive adversary case in 
Section IV-DI to allow for correcting the errors introduced by 
the adversary. 

A data collector accessing any k nodes will observe a total 
of ak = (n — \)k symbols, out of which M = Y^i=ii n — 
symbols have distinct indices, and k ( k ~ ls > symbols are re- 
peated due to the RSKR-repetition code. The adversary can 
corrupt identically the two copies of each symbol stored on 
the b controlled nodes. Therefore, the data collector focuses 
on M symbols with distinct indices out of (n — l)k and uses 
them for decoding. These M symbols with distinct indices 
form a codeword of an (M, R) MDS code, say X, which are 
possibly corrupted by the errors introduced by the adversary. 
The minimum distance of the MDS code X is, 

2b 

d min (X) = M-R + l = J2( n - i ) + 1 - (13) 

i=l 

The adversary that controls b nodes can introduce up to 
t = Y^i=i( n ~ *) errors in the set of M symbols with distinct 
indices. A simple manipulation shows that t > ^ dmm W — J.J 
Therefore, a classical minimum distance decoder for X will 
not be able to recover the original file. Thus, the minimum 
distance decoder fails for this specific adversarial strategy 
where Calvin corrupts the repeated symbols identically and 
cannot be used for a general adversarial strategy. 

Next, we present a novel decoder that can correct errors 
beyond the classical upper bound of [ d "" n ^' ) ~ 1 j in the DSS. 
The main idea is to take advantage of the special structure of 
the error patterns that can be introduced by the adversary. 

First, we introduce two definitions that will be useful in 
describing the decoding algorithm and that will serve as a 
generalization of the concept of trusted set in the previous 
example. 

Definition 8: Puncturing a vector: Consider a vector v £ 
¥ N for some field F. Let / C {1,2,..., N}, \I\ = p, be a 
given set. Then puncturing vector v with pattern / corresponds 
to deleting the entries in v indexed by the elements in / to 
obtain a vector vj € ¥ N ~ P . 

Definition 9: Puncturing a Code: Consider a code C in 
¥ N . Let I C {1,2,..., N}, \I\ = p, be a given set. The 
punctured code Ci is obtained by puncturing all the codewords 
of C with pattern /, i.e., 

Ci := {xi\x £ C}. 

Proposition 10: If C is an MDS code with parameters (n, k) 
then for any given fixed pattern / C {1, 2, . . . , n}, \I\ = p < 
(n — k + 1), the punctured code Ci is also an MDS code with 
parameters (n — p, k). 

Decoding Algorithm: Let B, \B\ < b, denote the set of 
storage nodes controlled by the adversary. Because of the exact 
repair property of the RSKR-repetition codes, it is sufficient 
to focus on the case when B C {i>i, . . . ,v n } with \B\ = b. 
For each such set B, we define Ib C {1,2,..., 6} to be the 
set of the indices of the symbols stored on the nodes in B. 
For instance, in Example[5] if B = {v±}, Ib = {1, 2, 3}. 

The decoding algorithm proceeds in the following way: 



1) The data collector connecting to k nodes selects any 
M symbols with distinct indices, out of the (n — l)k 
observed symbols, as its input Y G F^ 1 for decoding. In 
Example|5] Fig. [8] the DC connecting to nodes v±, v$, v§ 
observes vector (y lt y 2 , y 3 , Vi, 2/4, V5, 2/2, 2/4, Ve)- Af- 
ter removing the repeated symbols, we get Y = 
{yi, 2/2, 2/3, 2/4, 2/5, 2/6). Note for a fixed DC, F is a 
codeword of an (M, R) MDS code which we call X. 
Y includes possible errors introduced by the adversary. 
The code X itself is a punctured code of the outer (9, R) 
MDS code. 

2) For each B C {i>i, . . . , v n }, \B\ = b, find Ib- 

3) Puncture Y and the code X with pattern Ib to obtain 
the observed word Yj B and punctured code Xj B . Note 
that due to the RSKR-repetition structure, the size of 
such puncturing pattern is 

b 

i/ai = 5>-i) 
i=i 

which is less than the minimum distance of the MDS 
code X (see (TT3l). Hence, by Proposition [TOl Xr r is an 
MDS code. 

4) Let Hx Ib be the parity check matrix of the punctured 
code Xj B . Compute the syndrome of the observed word 

Y Ib as 

oi B =H Xl Yj B . 

5) If <7j B = 0, then Yj B is a codeword of Xj B . Assume it 
to be a trusted codeword and decode to message using 
the code Xj B . 

Proof of Correctness: We now prove the correctness of 
the above decoding algorithm by showing that it will always 
correct the errors introduced by the adversary and output 
the correct message. Notice first that the syndrome 3j B will 
always be equal to zero whenever B = B*, the actual set of 
nodes controlled by the adversary (which is not known to the 
data collector). Therefore, the above decoding algorithm will 
always give an output. Next, we show that this output always 
corresponds to the correct message stored on the DSS. Denote 
by X the true codeword in X, that would have been observed 
by the DC in the absence of Calvin. Let B* be the set of the b 
traitor nodes. Then, the proposed decoding algorithm fails iff 
there exists some other set B ^ B*, and some other codeword 
X' e X, s.t. X' ^ X, for which Y Ib = X' Ib £ X Ib . This 
implies that 

Xi B ,ui B = X i B ui B , ■ ( 14 ) 
But, from the RSKR-repetition code structure we know 

2b 

\Ib*UIb\ < X)( n- ')- (15) 

i=l 

Equations (fT4l) and dHl ) imply that d min (X) < X^=i( ?1 ~*) 
which contradicts equation ( fT3l . 

Remark 11 (Decoder complexity): The complexity of the 
proposed decoder is exponential in the number b of malicious 
nodes. Therefore, it is not practical for systems with large 
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values of b. However, this decoder can be regarded as a proof 
technique for the achievability of the resiliency capacity Cf L 
of Theorem [7j 

Remark 12: [Expurgation of malicious nodes] As shown 
above, the proposed decoder always decodes to the correct 
message, and thus, can identify the indices of any erroneous 
symbols. The data collector can then report this set of indices 
to a central authority (tracker) in the system. This authority 
will combine all the sets it receives, and knowing the RSKR- 
repetition structure (see Fig. |6), it forms a list of suspected 
nodes that will surely include the malicious nodes that are 
sending corrupted data to the data collectors. Since there are 
at most b malicious nodes and each symbol Xi is stored on 
exactly two nodes, the size of the list will be at most 2b. The 
system is then purged by discarding the nodes in this list. 

VII. Active Limited-Knowledge Adversary 

In this section, we consider the case of a non-omniscient 
active adversary with limited eavesdropping and controlling 
capabilities. We assume the adversary can eavesdrop on £ 
nodes and control some subset of b < £ nodes out of these 
£ nodes. The adversary's knowledge about the stored file 
is limited to what it can deduce from the observed nodes. 
Moreover, we assume that the adversary knows the coding 
and decoding strategies at every node in the system. Clearly 
when £ > k, the adversary becomes omniscient. We are 
interested here in the limited-knowledge scenario that does not 
degenerate into the omniscient model studied in the previous 
section. For this case, we demonstrate that the resiliency 
capacity of the DSS exceeds that of the omniscient case, and 
can be achieved by storing a small hash on the nodes in 
addition to the data. Our approach is similar to that of l23l . 
GU, ED, where the authors consider a limited-knowledge 
adversary that can eavesdrop and control edges rather than 
nodes in multicast networks. 

Example 13: Consider a DSS 2?(5, 3,4) with a = 7 = 4 
with an adversary Charlie that can eavesdrop on and control 
one node, i.e., b = £ = 1. In the omniscient case with 6=1, 
the resiliency capacity of this system as given by Theorem [7j 
is equal to 2. Here, we show that the limitation on Charlie's 
knowledge can be leveraged to increase the resiliency capacity 
to 5. 

First, we show that the resiliency capacity for this DSS is 
upper bounded by 5. To that end, consider the case when node 
V\ is observed and controlled by Charlie. Moreover, assume 
that nodes 1)2 and W3 fail successively and are replaced by 
nodes vq and v-j as shown in Fig. [9] Consider now a data 
collector DC that connects to nodes v\ , Ve, vj and wants to 
reconstruct the stored file. One possible attack that Charlie 
can perform, is to erase all the data stored on node v\, i.e., 
always change it to a fixed value irrespective of the stored 
file. This renders node v\ useless and the system performs as 
if node v\ was removed which reduces the value of the cut 
C(V, V) (see Fig. |9]l between the source s and data collector 
DC to 5. 

We now exhibit a code that uses a simple "correlation" 
hash scheme to achieve the above upper bound with high 
probability. 




Fig. 9. The limited-knowledge adversary Charlie eavesdrops and con- 
trols node vi, shown with the broken boundary. If Charlie erases the 
data stored on node vi, the value of the cut C(V,V), with V = 
{%o U t' x in' x ouf x Jn' ^out'DC}, between the source node s and a data 
collector DC accessing nodes vg,Vf,vg becomes equal to 5. 

a) Code Construction: The code consists of an outer 
(10, 5) MDS code over ¥ r , followed by the RSKR-repetition 
code enabling the exact repair of the nodes in the case of 
failures. Furthermore, each data packet Xi £ ¥ q v is appended 
with a hash vector hi = (/i^i, . . . , /ij,io) G computed as, 

for j = 1,2,..., 10, where with abuse of notation, Xi also 
denotes the vector (2^1, . . . , Xi, v ) in ^ v q representing the 
corresponding element of F g « . The schematic form of the code 
is shown in Table [Hi below. 

For simplicity, we assume in this example that the hash 
values stored on the nodes are made secure from Charlie who 
can neither observe, nor corrupt them. Later in Appendix ICl 
we explain how this can be achieved in the general case with 
a negligible sacrifice in the system capacity. Note that even 
though Charlie cannot directly observe the hash table, he can 
generate some of the hash values using the observed data 
packets on £ — 1 eavesdropped nodes, since he knows the 
coding scheme. Charlie can use these computed hash values 
to carefully introduce errors in the data symbols such that it 
is still consistent with these hash values. 



Node 


data e ¥ q v 


hash e F^ u 


VI 


Xl,X2,X 3 ,X4 


hi, h 2 , h 3 , h 4 


V2 


Xl,X5,X 6 ,X7 


hi, h 5 , h 6 , h7 


V3 


X2,X5,X8,X 9 


h 2 , h 5 , h 8 , hg 


Vi 


X3,X6,X 8 ,X10 


h 3 , h 6 , h 8 , hio 


V5 


X4,X7,X9,X10 


h4, h7, hg, hio 



TABLE II 

The schematic form of the code stored on the DSS X>(5,3,4), 

ALONG WITH THE SECURE HASH TABLE THAT IS NOT ACCESSIBLE TO THE 
ADVERSARY CHARLIE. 

b) Decoding logic: A data collector contacting 3 nodes 
observes 12 symbols in total. In a worst case scenario, Charlie 
can corrupt 6 out of these 12 symbols. This can happen, 
for instance, when Charlie eavesdrops and controls node v±, 
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and maliciously changes its stored data from Xi to yi = 
Xi + ej,ej 7^ 0, i = 1,...,4. Then, V2,v% fail successively 
(as shown in Fig. |9]l and Charlie sends the erroneous symbols 
yi and y2, respectively, to nodes v$ and vq during the 
repair process. In this scenario, a data collector, unaware of 
Charlie's actual node location, accessing nodes v\,vq and 
vj will have among its observation 6 corrupted symbols, 
namely those having indices 1, . . . ,4 as shown in Table [HI] 
where the symbol yi denotes the possibly corrupted version 
of Xi, i = 1, . . . , 9. Here, we have yi = Xi, i = 5, . . . , 9. The 
table also shows the hash vectors observed by the same data 
collector. 



Node 


data € ¥ q v 


hash e F^ u 


fi 


yi.y2,y3,y4 


hi, h 2 , h 3 , h4 




yi.y5,y6,y7 


hi, h 5 , h 6 , h 7 


V7 


y2,y5,ys,y9 


h 2 , h 6 , h 8 , h 9 



TABLE III 

The data symbols and hash values observed by the data 
collector contacting nodes vi , v 6 , v-j , when node vi is 
controlled by charlie. 

Among the 12 stored symbols Xi observed by the data 
collector and their hashes hi, each of the 3 symbols with 
indices 1,2,5 and the corresponding hash vectors h\,h<2,,h$ 
are repeated twice. Since the adversary can change both copies 
of each repeated data symbol identically, our decoder focuses 
only on a set of M = 9 symbols of distinct indices and 
the corresponding hash vectors for decoding. Note that the 
corresponding 9 symbols (xi, . . . , Xg) form a codeword of a 
(9, 5) MDS code that we refer to as X. 

Let H denote the 9x9 hash matrix observed by the data 
collector, obtained as 

" hi " 
h 2 

H= . , 
. h 9 . 

where the i' h row hi e corresponds to the hash vector of 
the symbol yi, i = 1, . . . , 9. The data collector then computes 
its own 9x9 hash matrix H from the 9 observed symbols yi 
as 

^y=yiyj T , i<i,i<9. 

Then, it compares the entries in H with the corresponding 
entries in H to generate a 9 x 9 comparison table. Table ITVl is 
an example of such a comparison table where a "/" in position 
(i, j) indicates that the computed hash and the observed hash 
match, i.e., Hij = Hij, whereas "x" indicates that Hij ^ H^ 
due to the errors introduced by the adversary. 

The decoder selects a trusted set of 5 symbols from 
{yi,...,yg} that index a 5 x 5 sub-table of the com- 
parison table where all the entries are "•/", e.g., symbols 
ys, Y6, Y7, ys, Y9 in Table ITVl It then sets the remaining 4 
symbols as erasures and proceeds to decode using a min- 
imum distance decoder for the (9, 5) MDS code X, that 
can correct up to 4 erasures. There always exists at least 
one set of 5 symbols that generates a consistent hash table, 
e.g., T = {y 5 ,ye,y7,y8,y9} when Charlie controls node 



Data Symbol 


yi 


y2 


y3 


y4 


ys 


Y6 


y7 


ys 


y9 


yi 


/ 


/ 


/ 


/ 


X 


X 


X 


X 


X 


y2 


/ 


/ 


/ 


/ 


X 


X 


X 


X 


X 


y3 


/ 


/ 


/ 


/ 


X 


X 


X 


X 


X 


y4 


/ 


/ 


/ 


/ 


X 


X 


X 


X 


X 


ys 


X 


X 


X 


X 


/ 


/ 


/ 


/ 


/ 


ye 


X 


X 


X 


X 


✓ 


/ 


/ 


/ 


/ 


y7 


X 


X 


X 


X 




/ 


/ 


/ 


/ 


ys 


X 


X 


X 


X 


✓ 


/ 


/ 


/ 


/ 


yg 


X 


X 


X 


X 




/ 


/ 


/ 


/ 



TABLE IV 

Example of the comparison table of the hash matrices H and 
H. Note that since Charlie observes the data symbols 

{xi, . . . ,X 4 }, HE CAN INTRODUCE ERRORS SUCH THAT THE HASH 
VALUES OF {yi . . . , y 4 } ARE CONSISTENT. 



V\. Hence, the proposed decoding will eventually stop and 
output a decoding decision. Next, we analyze the probability 
of selecting a trusted set that results in an error in decoding. 

c) Error Analysis: Let E = {xi,...,X4} denote the 
set of data symbols observed by Charlie by eavesdropping on 
I = 1 node (f i in this case). The above proposed decoder may 
result in an error only if the chosen trusted set T contains at 
least one erroneous symbol, say yi. Therefore, we can write 
yi = xi + ei for some error ei ^ G F q v . Any chosen 
trusted set T is also guaranteed to contain at least one error- 
free symbol that is not observed by Charlie, say ys = X5 ^ E. 
To see this, note that the cardinality of the trusted set T is 5, 
and by eavesdropping and controlling any one node Charlie 
can observe and introduce errors in a maximum of 4 symbols 
with distinct indices to any data collector observation. For the 
set T, containing yi , ys along with 3 other symbols, to be a 
trusted set, it has to generate a consistent hash table of size 
5x5. Therefore, Charlie has to pick the error ei to satisfy 
x 5 ei T = 0. 

The observation E = {xi,...,X4} of Charlie is inde- 
pendent of X5 due to the MDS property of the outer code. 
Therefore, for any choice of ei that Charlie makes, there 
are q v equally likely choices of X5, out of which q"~ 1 are 
orthogonal to the chosen ei. Hence, the consistency condition 
of hash = H$ i is satisfied with probability, 

P r (x 5 e 1 T = 0|i;,e 1 ) = i. 

Note that if Charlie could observe the complete hash table, 
then X5 is no more independent of Charlie's observation. For 
example, if Charlie observes the hash value H2.5 = X2Xs T , 
then for a given value of x 2 and H2.5, there are only q"^ 1 
equally likely choices for x 5 . In which case Charlie can 
always choose e! to belong to the space orthogonal to v — 1 
dimensional space of possible choices of X5, thus, deceiving 
the proposed decoder. Therefore, it is crucial to keep the hash 
values secure from Charlie. 

It can be verified that the above reasoning easily carries to 
any choice of b = 1 node controlled by Charlie. Therefore, the 
probability of error is upper bounded by 1/q which vanishes 
with increasing the field size q. 

d) Rate Analysis: We encode 5 information symbols in 
F q v to form the coded symbols Xi, i = 1, . . . , 10. For these 
10 symbols we construct a hash table of size 10 x 10 with 
elements in ¥ q . Hence the total overhead of the hash table 
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is ^2 — O(i) per information symbol. Thus, the rate of our 
code is 5 — O(-) which approaches 5 with an increasing block 
length v. 

A. Results on Active Limited-Knowledge Adversary 

Below we summarize our two main results on the resiliency 
capacity in the case of a limited-knowledge adversary. 

Theorem 14: For a DSS T>(n, k, d) with an adversary that 
can eavesdrop on any £ < k nodes and control a subset of size 
b of these £ nodes (b < £), the following upper bound holds 
on the resiliency capacity, 

k 

C r (a,<y)< min{(d-i+l)l3,a} (16) 

i=b+l 

where d/3 = 7. 

Proof: (sketch) Consider a case when nodes v\,...,Vk 
fail successively and are replaced by nodes v n +i, . . . ,v n +k 
as shown in Fig. [10] Also consider a data collector DC 
that contacts these k nodes {v„+i, ■ • ■ ,v n +k} to retrieve the 
source file. If the adversary Charlie controls the b nodes 
{v n +i, . . . , «„+&}, one possible adversarial strategy that Char- 
lie can use is to erase all the data stored on these b nodes, 
i.e., always change it to a fixed value irrespective of the file 
stored on the DSS. This renders the b controlled nodes useless, 
resulting in the u P£ er bound stated in the theorem. ■ 

Let R := J2i=b+i mm {(^ — i + a:} and £ := 
X)i=i mm {(^ — i + l)/3, a}. Our second results states that 
if the eavesdropping capability I of the adversary Charlie is 
limited, in particular i is such that £ < R, the upper bound in 
Theorem [14] can be achieved for d = n — 1 in the bandwidth- 
limited regime. 

Theorem 15: Consider a DSS T>(n, k,d = n — 1) operating 
in the bandwidth-limited regime in the presence of an adver- 
sary that can eavesdrop on I nodes and controls a subset of 
size b of these £ nodes (b < I). Then, if the adversary is 
limited-knowledge, i.e., £ is such that £ < R, the resiliency 
capacity of the system is, 

k 

C r BL (T) = J2 ( n ~ Oft 

i=b+l 

where (3 = T/(n - 1). 

The condition £ < R in Theorem Q3] says that the 
eavesdropping capability of the adversary is insufficient to 
determine the message stored on the DSS, i.e., the adversary is 
not omniscient. This limitation in the adversary's knowledge 
enables every data collector to identify the erroneous symbols 
introduced by the adversary and discard them, thus, resulting 
in erasures rather than errors. In this case also, identifying the 
erroneous symbols helps in the expurgation of the system and 
discarding the malicious nodes, as pointed out in Remark IT2l 

The proof of Theorem Q3] is detailed in Appendix [C] and 
is composed of two parts. In the first part, we assume that 
the hash table is secure from the adversary and generalize the 
reasoning of Example Qj] to show how the hash table can be 
used to identify, with high probability, the erroneous symbols 
introduced by Charlie and thus decode correctly. In the second 




Fig. 10. Part of the information flow graph corresponding to a DSS 
T>(n,k,d), when nodes vi,...,Vk fail successively and are replaced by 
nodes v n +i, • ■ • , fn+fc- A data collector contacts these k nodes and wants 
to reconstruct the stored file. Nodes u n _|_i , . . . , « n +< shown with broken 
boundaries are compromised by Eve while they were being repaired. 



part, we demonstrate an efficient scheme to store the hash table 
securely and reliably with a negligible sacrifice in the system 
capacity. 

VIII. Conclusion 

In this paper we have considered the problem of securing 
a distributed storage system under repair dynamics against 
eavesdropping and adversarial attacks. We proposed a new 
dynamical model for the intrusion, wherein the adversary in- 
trudes the system at different time instances in order to exploit 
the system repair dynamics to its own benefit. For the general 
model of an adversary that can eavesdrop and/or maliciously 
change the data on some nodes in the system, we investigate 
the problem of determining the secrecy capacity and resiliency 
capacity of the system. We provide upper bounds on the 
secrecy and resiliency capacity and show their achievability 
in the bandwidth-limited regime. General expressions of these 
capacities in addition to efficient decoding algorithms remain 
an open problem. 

Appendix 

A. Proof of Theorem [7] 

Consider a DSS T>(n,k,d) with £ < k, operating at 
point (a, 7) with df3 = 7. Assume that nodes v±, V2, ■ ■ ■ , Vk 
have failed successively and were replaced during the repair 
process by the nodes v n +i,v n +2, ■ ■ ■ ,v n +k respectively as 
shown in the corresponding information flow graph Q in 
Fig. [TO] Now suppose that Eve accesses the I input nodes 
in the set E = x?+ 2 , . . . , x^ 1 } C V in while 

they were being repaired. Consider also a data collector 
DC that downloads data from the k output nodes in B = 
{<it\x n tt, x n oit} G Volt- The reconstruction property 
of Eq. dU implies H(S\Cb) = and the perfect secrecy 
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condition in Eq. (0 implies H(S\De) = H(S). We can 
therefore write 

H(S)=H(S\D E )-H{S\C B ) 

< H{S\C E ) - H{S\C B ) 

= H(S\Ce) — H(S\Ce,C b \ e ) 

— I(S, C b \e\Ce) 

< H(C b \ e \Ce) 

fc 

— H (Cn+jICn+i, . . . , Cn+i-l) 

i=e+i 

(3) _ 

< V] min{(d-i + l)/3,a} 
i=f+l 

Inequality (1) follows from the Markov chain S — > 
De -> Ce i.e., the stored data C B is dependent on S 
only through the downloaded data D B , (2) from C B \ B '■= 
{C n +e+i, . . . , C„-|_fc}, (3) follows from the fact that each node 
can store at most a units, and for each replacement node we 
have H(Ci) < H(Di) < d/3, also from the topology of the 
network (see Fig. ITOb where each node x™^ 1 is connected to 
each of the nodes x™^ 1 , . . . ,a;™^"/ _1 by an edge of capacity 
/3. The upper bound of Theorem Q] then follows directly from 
the definition of Eq. ©. 



C{V,V) 




Fig. 11. Part of the information flow graph corresponding to a 
DSS (n,k,d) when nodes Vj+i,...,Vk fail successively and are re- 
placed by nodes u n -|_i, . . . , v n +k—j. A data collector connects to nodes 
vi, . . . ,Vj, Dn+i , ■ ■ ■ , j to retrieve the file. 



B. Proof of Theorem [6] 

Consider a DSS T>(n,k,d) operating at point (a, 7) 
with dj3 = 7, in the presence of an omniscient adver- 
sary that can control b nodes, with 2b < k. Assume 
that nodes Vj+i,Vj+2, . . . ,Vk, for some j, 2b < j < 
k, have failed consecutively and were replaced by nodes 
v n +i,V n +2, ■ ■ ■ ,v n +(k-j)> respectively. The information flow 
graph Q of the DSS corresponding to this sequence of 
node failures and repairs is shown in Fig. [TT] Consider a 
data collector (Fig. ITTb that observes the stored data on 
the k nodes V\, . . . , Vj,v n +i, ■ • ■ , v n+ k-j- Consider also the 
cut C(V,V) with V = {xl ut ,...,4 ut ,x?+\...,x% k - j , 
x™^ 1 , . . • , x^ut J I DC} that separates the source node s from 
the data collector DC. We group the edges belonging to this 
cut into 3 disjoint sets as follows: 

1) E\: the set of edges outgoing from nodes x? n ,p = 
l,...,b. 

2) E2' the set of edges outgoing from nodes xf n ,p = b + 
I,. ..,2b. 

3) E3: the set of edges outgoing from nodes x? n ,p = 2b + 
1, . . . , j, in addition to the edges belonging to the cut 
C(V, V) that are incoming to the nodes xf n ,q = n + 
l,...,n + k-j. 

Let X^(m),i = 1,2,3, be the symbols transmitted on the 
edges in set Ei corresponding to the stored message m. We 
claim that in the presence of an adversary controlling any 
b nodes and for any two distinct messages mi 7^ the 
following condition is necessary for the DC to not make a 
decoding error: 

X Es {mi) ^ X E3 (m 2 ). 



Suppose that there exist two distinct messages mi ^ ni2 
satisfying X Es {mx) = Xg 3 (777,2)- Now, if the symbols 
carried on the edges belonging to the cut C(V, V) are 
XE 1 {mi),X E2 {m 2 ) and ^(mi) = X E3 (m 2 ). Then, as- 
suming all the messages to be equally likely, the data collector 
will make a decoding error with probability at least 1/2. This 
is true since it will not be able to distinguish between the 
following two cases: 

« The true message is 7772 and the nodes x\ n , . . . , x\ n 
are controlled by the adversary Calvin who changed the 
transmitted symbols on the edges in the set E\, from 
XE 1 {m 2 ) to X El (mi). 

• The true message is mi and the nodes x^ 1 , . . . , xf^ 
are controlled by the adversary Calvin who changed the 
transmitted symbols on the edges in the set E 2 , from 
X E2 {mi) to X E2 {m 2 ). 

Thus, the capacity of the DSS is upper bounded by the total 
capacity of the edges in the set E3, i.e., 

j fc 

CV(a,7)< Yl a+ (d-i+l)0, 3 = 26+1,..., fc-1. 

i=2b+l i=j + l 

The same analysis, as above, can be applied for j = 2b 
resulting in, 

fc 

C r (a,j) < (d-i + l)p. 

i=2fc+l 



And also for j = k, which gives, 

fc 

C r (a,j) < ^2 a. 

i=2b+l 

The bound in Theorem [6] then follows by taking the minimum 
of all the above upper bounds obtained for j = 2b, . . . , k. It 
can be easily seen that the above argument extends to the case 
of 26 > k for which the set E3 is empty and C r (a,7) = 0. 
□ 
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data packet £ F g u 


hash e Fj 


VI 


xi 


X2 




Xn-1 


hi 


h 2 




hn-l 


V2 


xi 


Xn 




X2n-3 


hi 


h„ 




h2n-3 


V3 


X2 


Xn 




X3n-6 


h 2 


h„ 




h3n-6 




















Vn 


Xn-1 


X2n-3 




x e 


hn-l 


h2n-3 




he 



TABLE V 

Schematic form of the code stored on the DSS (n,k,d = n- 1), 

ALONG WITH THE HASH TABLE THAT IS NOT ACCESSIBLE TO THE 
ADVERSARY CHARLIE. 



C. Proof of Theorem 1751 

Consider a DSS V(n, k, d), with d = n— 1, operating in the 
bandwidth-limited regime, in the presence of an adversary that 
can eavesdrop on £ nodes and control a subset of them of size 
b, b < £. As in the earlier proofs, we show the achievability 
for (3 — 1, i.e., T = n — 1. Any larger values of f3 or T 
can be achieved by repeatedly applying the proposed scheme. 
Since there is no constraint on the node storage capacity a in 
bandwidth-limited regime, we choose a = n — 1. Let := 
M := E-=i(n ~ i). R Et b+ i(n - i) and £ := 

Our proof consists of two parts: 1) We assume that the 
hash table can be stored securely and reliably, and show an 
achievable scheme that can attain the resiliency capacity. 2) We 
present an efficient method to reliably and securely store the 
hash table in the presence of a limited-knowledge adversary 
Charlie. 

C.l Resiliency Capacity in the Limited-knowledge Case for 
the Bandwidth-Limited Regime 

Code Construction: The code that we propose here is a 
generalization of the one used in Example Q~3] of Section I VIII 
It consists of an outer (9, R) MDS code whose output X = 
(xi , . . . , xg ) £ V 6 qV is stored on the n storage nodes using an 
inner RSKR-repetition code that enables exact repair in case 
of any node failure. As shown in Table [V] each data packet 
Xi 6 ¥ q v , i = 1, . . . , 9, is further appended with a hash vector 
hi = • • • ) hifi) £ F®. The values of these hashes are 

computed as follows, 

for j = 1,2, ... ,9, where with abuse of notation Xi also de- 
notes the vector in F^ representing the corresponding element 
of Wqv. We assume for now that the hash values stored on 
the nodes are secure from Charlie who can neither observe 
nor corrupt them (as shown in the next section). Although 
Charlie cannot directly observe the hash table, he can compute 
some of the hash values using the observed data packets on 
I eavesdropped nodes and possibly introduce errors that are 
consistent with these hash values. 

Decoding Logic: A data collector accessing any k nodes 
will observe a total of (n— l)k symbols and the corresponding 
hash vectors, where (z\ indices are repeated twice. As noted 
earlier, since the adversary can corrupt both of the stored 
symbols with same indices identically, the decoder focuses 
only on a set of M = X)i=i( n — symbols with distinct 



indices along with their hash vectors to make a decoding 
decision. These M symbols form a codeword of an (M, R) 
MDS code X possibly corrupted by errors introduced by the 
adversary. 

Recall that Charlie can eavesdrop on a total of £ nodes and 
control some subset b < £ of these eavesdropped nodes in 
the system. Let yi, i = 1, . . . ,9, denote the possibly corrupted 
version of the original data symbols Xi. We have yi = Xi +ei, 
where ej is the error introduced by Charlie on the symbols 
stored on the nodes he controls, and for rest of symbols 
ei = 0. Without loss of generality, we suppose that the 
data collector observes nodes vi,...,Vk, i.e., data symbols 
yi and hash values hi,i £ {1,2, ...,M}. The data collector 
observes the hash values with no errors since the hash table 
is assumed to be secure and reliable against the adversary. 
Let H denote the observed M x 9 hash matrix having the 
vectors hi 6 F^,z = 1,...,M as rows. The data collector 
then computes its own M x M hash matrix H as 

^■=yiyj T , i<i,j<M 

from the observed M data packets and compares it with the 
corresponding entries in H . It generates anMxM comparison 
table similar to Table [TV] in Example [13] In this table a "•/" 
in the z-th row and j-th column indicates that the computed 
hash and the observed hash match, i.e., Hij = Hij, whereas 
"x" indicates that Hij ^ Hij due to the errors introduced by 
the adversary. 

The decoder then selects a set of R symbols, among 
(yi, . . . , yjvi), that index an R x R sub-table of the 
comparison table with all its entries equal to and 
declares it as a trusted set with no errors. Then, it sets the 
rest of the M — R observed symbols as erased and proceeds 
to decode the obtained vector as a codeword of an (M, R) 
MDS code X with M — R erasures. Since Charlie can 
control only b nodes there always exists at least one set 
of size M — Y^,=i( n — i) ~ R symbols that generates a 
consistent hash sub-table of size Rx R with "•/". Hence, the 
proposed decoder is guaranteed to stop. Next, we compute 
the probability that the above decoder decodes to an incorrect 
message. 

Error Analysis: The proposed decoder may result in an error 
in decoding only if the chosen trusted set of R observed sym- 
bols contains at least one erroneous symbol, say yj = xj +ej, 
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ej ^ 0. Also, since b < I, we have, 
b e 
Y,(n-i)<Y,( n - i 1 <R > (18) 

i=l i=l 

where the last inequality follows from our assumption (see 
Theorem |T3T > that the eavesdropping capability £ is strictly 
less than the desired storage rate R. From equation < fT~8T >, it is 
clear that the chosen trusted set contains at least one error-free 
symbol that is not observed by Charlie, say y, = xi ^ E. For 
this set to be a trusted set, it has to generate a consistent hash 
table of size R x R. In particular fly = fly, i.e., Xiej T = 0. 

Next, we compute the probability of such event. Let E be 
the set of symbols in the codeword X that are observed by 
Charlie. Since X is the output of a (9, R) MDS code and 
\E\ < R, any symbol Xj of X that does not belong to E is 
uniformly distributed in W q v conditioned on E, i.e., 

Pr(x i = x i \E) = \, x«eF,„. (19) 
1 

Therefore, for any choice of ej that Charlie makes based 
on his observation E, there are q v equally likely choices of 
Xi out of which q 1 '" 1 are orthogonal to the chosen ej. Hence, 
the consistency condition of hash fljj = fljj is satisfied with 
probability, 

P r (x i e j T = 0|S,e j ) = i 

which goes to zero with increasing field size q. 

Note that if Charlie could observe the complete hash table, 
Xj would no more be independent of Charlie's observation. 
Then, as shown in Example [T3l Charlie can always choose ej 
to belong to the orthogonal space of all possible choices of Xi, 
thus deceiving the proposed decoder. Therefore, it is crucial 
to keep the hash values secure from Charlie. 

Rate Analysis: We encode R information symbols in ¥ q v 
using a (8, R) MDS code to form a codeword (xi, . . . , xg). 
For these symbols we construct a hash table of size 8x6 with 
symbols in ¥ q . Hence the total overhead of the hash table 
is = O(^) per information symbol which goes to zero 
with increasing block length v. Hence, asymptotically in block 
length v, these codes achieve the capacity of Theorem Q3] 

C.2 Reliable and Secure Storage of the Hash Table 

The scheme described here for storing the hash table 
securely and reliably is along the parallel lines of the scheme 
proposed |23F in the context of securing multicast networks. 
It aims at storing 1 bit of information securely and reliably. 
The scheme can then be repeated to store the complete 
hash table which, as shown in the previous section, is of 
constant size and independent of the block length v of 
the information symbols. The total overhead incurred by 
this scheme can be then made arbitrarily small by increasing v. 

5 The scheme of 1251 is matrix-based and is designed for networks where 
intermediate nodes perform random network coding. Our scheme here can be 
regarded as a simple vector version of the one in 1251 . This simplification 
is possible due to the special structure of the networks (information flow 
graphs) representing distributed storage systems in conjunction with the 
RSKR-repetition codes that limit coding in these networks to the source. 



Code Construction: Let G = [ „ be a generator 

\ G S J 

matrix of a (6, M) nested MDS code over the finite field ¥ q 
(symbols in the hash table also belong to the same field). The 
matrix Gk in itself is a generator matrix of a (0,£) MDS 
code over W q . If the bit to be stored is "1" then choose a 
vector S randomly and uniformly from ¥ q I ~ £ , otherwise, set 
S = G Ff - £ . Let JC = (K 1 ..., K £ ) denote £ random keys 
mutually independent and each uniformly distributed over ¥ q . 
Now, we form the vector X GF q to be stored on the DSS as 
part of the hash table by "mixing" S with the random keys 
using the nested MDS code as, 

X = KLGk + SGs- 

This encoded vector X G F q is then stored on the (n, k, d) 
DSS using the RS ICR-repetition code as shown in Fig. [6] The 
RSKR-repetition structure allows the exact repair of a node 
in case of failure as explained in Section [V] 

Security Analysis: The coding scheme used here is same as 
the one in Section IV-DI that discusses passive adversary and 
hence the vector S, which is of the appropriate rate M — £, is 
perfectly secure from Charlie eavesdropping on £ nodes. The 
perfect secrecy of S implies the perfect secrecy of the hash 
bit. 

Next we describe a decoding algorithm that the data 
collector uses to decode the stored bit with high probability 
of success even in the presence of errors introduced by 
Charlie controlling b nodes. 

Decoding Logic: We denote by B the decoder used by the 
data collector to recover the stored bit belonging to the hash 
table. B implements the same decoding steps as the decoder of 
Section [VI-BI of omniscient adversary, except for the decision 
rule that determines the output. The input to B is the data 
observed by the data collector accessing k nodes which is 
formed of ka = k(n — 1) symbols, among which („) pairs 
have the same indices. The decoder executes the following 
steps: 

1) B selects any set of M symbols having distinct indices 
among the observed ka symbols. These symbols are 
grouped in a vector Y G F^ 1 which can be written as 

Y = JCG K + SG s + e, 

where Gk and Gs are submatrices of Gk and Gs of 
size £ x M and (M — £ ) x M, respectively. The vector 
e G ¥ q ! , with up to Y^i=i( n ~ *) non-zero terms, is the 
error vector that accounts for the errors introduced by 
the adversary. 

2) Let B, \B\ = b, denote the set of storage nodes con- 
trolled by the adversary. Again, due to the exact repair 
property of the RSKR-repetition code it is sufficient to 
consider B C {v i, ...,«„} with \B\ = b. For each such 
set B, let Ib C {1,2,..., 9} denote the set of indices 
of the symbols stored on the nodes in B. 

3) For each possible B C {t>i, v-2, ■ . ■ , v n }, \B\ = b, B 
punctures Y with pattern Ib to obtain Yj B as 

Y Ib = K,G KlB +SG s , b +ej B , 



18 



where GV, and Gs, are the submatrices of Gk and 
Gs obtained by deleting the columns corresponding to 
the punctured elements of Y, and e/ B is the punctured 
error vector. 

4) D checks whether Yj B is a valid codeword of the code 
generated by the matrix Gk Ib by checking whether the 
corresponding syndrome is zero. 

5) The decoder D repeats steps 3) and 4) for each of the 
(™) sets B until the syndrome obtained in step 4) is 
zero. In this case, D declares that bit "0" was stored. 
Otherwise, if for all possible values of B no zero 
syndrome is obtained, ID) declares that "1" was stored. 

Error Analysis: We do the error analysis of the above 
decoding logic considering two different cases based on the 
value of the stored hash bit. 

• Hash bit '0': We will show that when the stored infor- 
mation bit is '0', the decoder D makes no error. In fact, 
this case corresponds to S = and, thus, Y = ICGk + e. 
Let B* be the actual set of nodes controlled by Charlie. 
Then, there is at least one set B = B* for which 
Yt* = ICGk » , since er B , = 0. As a result, the decoder 
always outputs "0". 
> Hash bit '1 ': Information bit ' 1 ' corresponds to 

Y = KG K + SG s +e, 

where (JC, S) is a uniformly random vector in F^ 1 and 
e 6 ¥ q J is the error vector introduced by Charlie. Note 
that the matrix G is a generator matrix of a (6, M) MDS 

G_K 

. G s 

invertible. Thus, we can write 

Y = ()C + eK)G K + (S + e s )G s , (20) 

where bk , es are the coefficients of the error vector e in 
terms of the basis corresponding to the rows of Gk, Gs- 
We have already shown in the security analysis above, 
that S is perfectly secure from Charlie's observation. 
Hence S + eg is a uniformly random vector in ¥ q vI ~ £ . 
Consider any set B C {v-y, . . . , v n } of cardinality \B\ = b 
with index set Is- Then, \Ig\ = Y^i = i( n ~ 0> hence 
the matrix Gj B obtained by deleting the columns of G 
corresponding to the indices Is has R = M — \Ib\ or 
more columns. Now, the matrix Gk z is a generator of an 
(M, £) MDS code and £ < R (Theorem [Bl Hence, the 
rank of Gk Ib is £ ■ This, along with the fact that G is an 
invertible matrix, implies that the rank of matrix Gs Ib 
is R — £ or more. The probability, that the syndrome 
computed in the step 4) of the proposed decoding logic 
for this set B is equal to zero, is equal to the probability 
of the event that a uniformly random vector (S + es) lies 
in the space orthogonal to the span of columns of Gs Ig ■ 
This probability is upper bounded by l/q R ~ £ . 
Now applying the union bound to all choices of the 
set B that the decoder attempts, the probability of error 
can be upper bounded by, 



code, hence the M x M sub-matrix G :- 



lim 



1 R-S 







which goes to zero with increasing the field size q. 

Rate Analysis: In the code proposed above to store the hash 
values securely and reliably we need 6 symbols in F g for 
each 1 bit of hash information. Also, in the previous section 
we showed that the total size of the hash table of interest is 
2 symbols in ¥ q . Thus, the total overhead of the proposed 
code to store the hash table is 6 3 logq symbols of ¥ q , that is 
independent of the block length v of information packets. 

Thus, we have shown how the hash table described in 
Table [V] can be stored on the DSS with a negligible overhead 
and is guaranteed with a high probability to be secret and 
resilient to the adversary provided that field size q and block 
length v are large enough. 
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Notation 


Explanation 


r> 
y 


Information flow graph of a distributed storage system. 


** j 

V 


Set of nodes in the information flow graph. 


C(V, V) 


Cut partitioning the set of nodes V in a graph into two sets V C V and V = V \V . 


s 


Random variable representing an incompressible source file. 


n 


Total number of active nodes in a distributed storage system. 


k 


Number of nodes a data collector connects to in order to retrieve the source file. 


d 


Number of nodes a new replacement node connects to during the repair process. 


a. 


Storage capacity at each storage node in a distributed storage system. 


a 

p 


Amount of data downloaded from every node participating in the repair process. 


7 


The total amount of data downloaded during the repair process i.e., repair bandwidth. 


r 


Upper limit on the repair bandwidth in the bandwidth-limited regime. 


Di 


All the data\messages downloaded on the replacement node Vi during the repair process. 


Ci 


Data stored on the node Vi. 


R 


Desired or achieved storage rate. 


M 


Capacity of the distributed storage system in the absence of an adversary. 


Xi 


Data symbol or packet stored on a distributed storage system. 


Yi 


Data symbol or packet, possibly corrupted by an adversary, observed by a data collector. 


I 


Number of nodes an adversary can eavesdrop on in a distributed storage system. 


b 


Number of nodes an active adversary can maliciously control. 


E 


A set of symbols\nodes observed by an adversary by eavesdropping on £ nodes. 


Cs 


Secrecy capacity of a distributed storage system. 




Resiliency capacity of a distributed storage system. 
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